Comparing the Interobserver Reliability of 4 Methods Used to Measure Knee Laxity on Coronal Plane Stress Radiograph

Background: Varus and valgus knee stress radiographs provide valuable information in the pre- and postoperative evaluation of joint laxity in patients with multiligament knee injuries (MLKIs). Purpose: To review the literature for described techniques of quantifying laxity on coronal stress radiographs of the knee and identify the most reliable method. Study Design: Cohort study (diagnosis); Level of evidence, 3. Methods: A thorough literature search using the MEDLINE and Embase databases identified 4 studies with distinct methods for objectively measuring laxity on varus and valgus stress radiographs: Heesterbeek et al (2008), Jacobsen (1976), LaPrade et al (2004), and Sawant et al (2004). To compare these methods, 200 coronal plane stress radiographs from 50 patients with MLKIs were retrospectively reviewed from an MLKI database at a single institution. The amount of varus and valgus laxity on each radiograph was measured independently by 4 reviewers using each method. Intraclass correlation coefficients (ICCs) with 95% CIs were calculated to assess the interobserver reliability of each method overall and the varus and valgus measurements individually. Results: For all 4 methods, the overall interobserver reliability was considered at least moderate. The method by Heesterbeek et al proved to have the highest interrater reliability in all domains—overall (ICC, 0.87 [95% CI, 0.85-0.90]), valgus (ICC, 0.83 [95% CI, 0.78-0.88]), and varus (ICC, 0.87 [95% CI, 0.83-0.90])—demonstrating good to excellent reliability both overall and in varus measurements and showing good reliability in valgus measurements. The method by Sawant et al demonstrated good reliability in valgus measurements. All other measures demonstrated moderate reliability. Conclusion: Available methods for measuring knee joint laxity on varus and valgus knee stress radiographs in patients with MLKIs demonstrated moderate to good interobserver reliability. The method described by Heesterbeek et al proved to have the highest reliability overall as well as in measurements on varus and valgus views individually.

Multiligament knee injuries (MLKIs) are complex orthopaedic injuries resulting from disruption to at least 2 of the major stabilizing ligamentous structures of the knee. 3,18These injuries are rare, representing an estimated 0.02% to 0.2% of all orthopaedic injuries. 2,3Clinical assessment of ligamentous laxity represents a critical step in the workup and diagnosis of MLKI, evaluation of surgical reconstruction success, and monitoring instability in conservative treatment protocols. 7,8,15While physical examination remains a mainstay of quantifying laxity in patients with MLKIs, it is prone to subjectivity.Stress view radiographs, however, allow visualization of laxity and the resultant joint space opening in an objective manner.
Stress view radiographs of the knee can be obtained while clinicians apply manual stress or automated force using a stress testing device in the coronal or sagittal plane. 15For coronal stress radiographs, the amount of medial and lateral joint space opening during valgus and varus stress testing of MLKIs, respectively, can be measured and compared with the unaffected knee.This provides clinicians with an objective and standardized evaluation of ligamentous laxity in patients with MLKIs.Any notable side-to-side differences can provide clinicians with valuable information regarding the stability of MLKIs, which can guide management and inform prognosis. 8Surgical indications are often based on these stress view assessments and other factors. 16Postoperatively, side-to-side differences can be monitored with serial stress radiographs to evaluate the integrity of operative treatment throughout the postoperative surveillance period.
There is significant heterogeneity in multiple dimensions of stress radiography-including force application, radiograph view, and joint angle. 8Previous authors have described varying techniques to objectively quantify joint laxity during the evaluation of valgus and varus stress radiographs. 6,7,12,17These techniques utilize different anatomic landmarks to measure the space or angle created between corresponding joint surfaces when valgus and varus forces are applied to the knee joint during stress radiography. 6,7,12,17While these methods have demonstrated strong reliability in previous studies, no investigation has compared the reliability of these measurement protocols against one another. 5,12,13As a result, no gold standard of varus and valgus stress radiography measurement has been identified.This has led to implementing heterogeneous methodology in clinical practice and research.Using a single measurement protocol with reliably consistent results may yield more comparable results between studies and facilitate the synthesis of the best available evidence.
The present study aimed to review the literature for described techniques of quantifying laxity on coronal stress radiographs of the knee and evaluate the accuracy and reliability of the different measurement techniques in patients with MLKIs in an effort to arrive at a recommendation for the most reliable method to use.

Literature Review
The study protocol received institutional review board approval.After a thorough literature search using the MEDLINE and Embase databases from inception until June 2020, we identified studies that have described 4 distinct methods for objectively measuring knee laxity on varus and valgus stress radiographs: Heesterbeek et al, 6 Jacobsen, 7 LaPrade et al, 13 and Sawant et al. 17 In the present study, these 4 studies are referred to as the Heesterbeek, Jacobsen, LaPrade, and Sawant methods.The search strategy utilized is summarized in Table 1.
The Heesterbeek method measures the angle between a tangent line on the subchondral bone of the femoral condyles and a line between the deepest tibial joint surfaces (Figure 1).Using the Sawant method, a tangent line is drawn at the subchondral bone of the femoral condyles.From this line, the distance of a perpendicular line drawn to the most medial or lateral point of the medial or lateral tibial plateau, respectively, is recorded (Figure 2).In the LaPrade method, the distance from the most distal aspect The authors have declared that there are no conflicts of interest in the authorship and publication of this contribution.AOSSM checks author disclosures against the Open Payments Database (OPD).AOSSM has not conducted an independent investigation on the OPD and disclaims any liability or responsibility relating thereto.
Ethical approval for this study was obtained from Unity Health Toronto (ref No. 12-027).
of the subchondral bone of the medial or lateral femoral condyle to the corresponding articular surface on the tibia is recorded (Figure 3).With the Jacobson method, 1 line is drawn tangential to the most distal portion of the medial and femoral condyles, and another line tangential to the most proximal subchondral bone of the medial and lateral tibial plateaus is then drawn.One line tangential to the lateral or medial border of the proximal tibia is drawn perpendicular to the tibial line.The distance of this vertical line between the femoral and tibial horizontal lines is recorded (Figure 4).

Radiographic Evaluation
Coronal plane stress radiographs from patients with MLKIs were reviewed from a large prospective MLKI database at a single institution.Each patient had undergone bilateral knee stress radiography with both varus and valgus stress.Measurements were completed on both injured and uninjured limbs to increase the number of radiographs reviewed and replicate the typical radiographic protocol at our institution, where the joint space opening of an injured knee is compared with the uninjured side.The maximum manual varus and valgus stress at 20°o f flexion was provided by the senior author (D.B.W.) while radiographs were captured.All patients with an adequate series of bilateral postoperative coronal plane stress radiographs were considered for study inclusion.The exclusion criteria were the presence of hardware that obstructed    Using each method, the amount of varus and valgus laxity on each radiograph was measured once independently by 4 reviewers with different levels of experienceincluding a medical student (G.W.S.), a junior orthopaedic resident (T.M.H.), a senior orthopaedic resident (S.Y.H.), and a clinical fellow (F.A.H.).Each reviewer was provided with a detailed training document that defined each method.Sample radiographs labeled with measurements were provided to consolidate training.Each reviewer was given a series of radiographs to complete measurements.One reviewer was unable to complete measurements on 3 radiographs.All measurements were completed using Carestream Vue Motion Software.

Statistical Analyses
Descriptive statistics were calculated using means and standard deviations.Intraclass correlation coefficients (ICCs) with 95% CIs were calculated based on an absolute agreement, and a 2-way random-effects model was used to assess the interobserver reliability of each method.The reliability of a single observer (ICC, 2,1) was examined.ICCs were calculated for varus and valgus measurements individually, along with a total ICC representing the pooled reliability from both valgus and varus measurements. 12ICCs ranged from 0 to 1, with higher ICCs indicating greater similarity between values.ICC values were interpreted according to guidelines from Koo and Li 11 : ICCs 0.90 indicated excellent reliability, 0.90 .ICCs .0.75 good reliability, 0.75 .ICCs .0.50 moderate reliability, and ICCs 0.50 poor reliability.All statistical analyses were conducted using SPSS Version 28.0 (IBM).

RESULTS
A total of 200 coronal plane stress radiographs from 50 patients with MLKIs-37 men and 13 women, with a mean (6SD) age of 44.12 6 11.79 years-were included in the study.The mean total joint opening for all radiographs measured by the 4 observers ranged from 6.16°6 2.70°to 6.58°6 2.77°using the Heesterbeek method; 13.25 6 3.60 mm to 16.54 6 3.55 mm using the Jacobsen method; 12.64 6 3.44 mm to 14.26 6 2.66 mm using the LaPrade method; and 12.64 6 3.44 mm to 14.26 6 2.66 mm using the Sawant method (Table 2).

DISCUSSION
Coronal plane stress radiographs objectively measure laxity and supplement clinical examination and aid in 4 Schemitsch et al The Orthopaedic Journal of Sports Medicine surgical decision-making for patients with MLKIs.In this study, we evaluated 4 different measurement techniques identified in a literature review.We found that the method described by Heesterbeek et al 6 proved to be the most reliable and accurate for quantifying medial and lateral joint laxity.However, no previous direct comparison between these methods has been conducted.A lack of consensus in MLKI knee stress radiograph measurements has led to the implementation of heterogeneous methodology in the literature.MLKIs are rare injuries that are particularly difficult to consistently evaluate with physical examination and magnetic resonance imaging.Stress radiography provides an objective measure of joint laxity and injury stability.Previous cadaveric studies have demonstrated the potential role of stress radiography in the diagnosis of high-grade varus and valgus knee injuries. 1,13However, without an accepted gold standard method for measuring laxity, it is difficult to make recommendations for diagnostic values or thresholds in the workup of these patients.Identifying a single accurate measurement method would allow for threshold laxity guideline recommendations, which could facilitate decision-making in the management of these patients before surgery and during postoperative follow-ups.
Previous studies have evaluated the reliability of these methods with strong reliability.Interobserver reliability (ICCs) for the LaPrade method, 5,9 Jacobsen method, 10,13,14 and Sawant method 4,17 have been reported in available studies to be 0.83 to 0.98, 0.87, and 0.95 to 0.99, respectively.The Sawant method has only been utilized to measure valgus stress radiographs in the literature.The present study demonstrated higher reliability with valgus stress using the Sawant method compared with varus stress.Furthermore, the reliability of the Heesterbeek method has not been reported in any previous research.These previously reported findings demonstrated higher reliability when compared with the present study.This may be secondary to utilizing multiple observers from multiple levels of training in the present study, which simulates the clinical environment of many academic institutions.Trainees with lower levels of orthopaedic knowledge may identify landmarks with less accuracy.In the present study, all observers underwent training before commenting on measurements to maximize their knowledge of radiographic landmarks.The Heesterbeek method demonstrated the highest reliability in all measures and may be more suitable for measuring joint laxity in future trials.
Of the 4 methods evaluated in this study, the Heesterbeek method is the only one that relies on angular measurement of medial or lateral joint space widening.In this method, tangent lines are drawn at the distal aspects of the femoral condyles and deepest portion of the tibia articular surfaces.These landmarks can be clearly identified radiographically, leading to accurate and reliable measurements of joint laxity in coronal stress radiographs.The Sawant, LaPrade, and Jacobsen methods rely on measuring vertical distance between the affected femoral condyle and the tibial plateau.This can affect the accuracy and reliability of the measurement, given the potential for variability in the form of start point, end point, and course of the vertical line measured.These techniques may be more prone to interobserver differences when there are rotational deviations in radiograph acquisition.Further research should be conducted to identify how the radiographic rotational profile affects the accuracy of these measurements.Moreover, on clinical examination, surgeons grade MKLIs according to the amount of joint space opening, similar to the Sawant, Jacobsen, and LaPrade methods, as opposed to grading injuries using angular measurements.Therefore, clinicians may gravitate toward these linear measurements, given their compatibility with physical examination findings.

Limitations
We recognize the limitations of the present study.There was variability in devices utilized to complete measurements by the individual raters.Because of the COVID-19 pandemic, some raters completed measurements remotely, creating screen size and resolution variability.Moreover, 1 reviewer was unable to complete measurements for 3 radiographs.Nonetheless, considering the large number of radiographs measured, this would have a limited impact on the results of the present study.In addition, the Sawant method had been described from the measurement of valgus stress radiographs.Our group applied the anatomic principles of the technique to varus radiographs with comparable reliability.Including raters with varying levels of training is both a strength and a limitation of the present study.The results of the present study may be more generalizable to academic centers where medical students, surgical residents, and fellows populate the clinical environment.Last, our study did not evaluate validity.Further research is needed to correlate the degree of laxity seen with each stress technique with clinical measurements of laxity, intraoperative findings, and patientreported outcomes to determine threshold values of clinically significant joint laxity obtained through stress radiography.

CONCLUSION
Available methods for quantifying laxity on varus and valgus knee stress radiographs in patients with MLKIs demonstrated moderate to excellent interobserver reliability across the 4 methods tested.Overall, the method described by Heesterbeek et al proved the most reliable in quantifying laxity in all directions in a large prospective MLKI database.Therefore, we recommend the Heesterbeek et al method for measuring laxity on coronal plane stress radiographs.However, further research is needed to determine and compare the validity of each method.

Figure 1 .
Figure1.The method of Heesterbeek.6The angle between a tangential line on the subchondral bone of the femoral condyle and a line through the deepest tibial plateau is measured.This value is then compared with the contralateral side.(A) Valgus measurement.(B) Varus measurement.

Figure 2 .
Figure 2. The method of Sawant. 17A tangential line on the subchondral bone of the femoral condyle is drawn.From this line, a perpendicular line to the most medial or lateral point of the medial or lateral tibial plateau is drawn, respectively.This value is then compared with the contralateral side.(A) Valgus measurement.(B) Varus measurement.

Figure 3 .
Figure3.The method of LaPrade.13The perpendicular distance from the subchondral bone surface of the most distal aspect of the femoral condyle and the corresponding tibial plateau is measured on both medial and lateral compartments on valgus and varus stress testing, respectively.(A) Valgus measurement.(B) Varus measurement.

Figure 4 .
Figure 4.The method of Jacobsen.7One line is drawn tangential to the most distal portion of the medial and lateral femoral condyle, and another line is drawn tangential to the subchondral bone of the medial and lateral tibial plateaus.Subsequently, a final line tangential to the medial and lateral border of the proximal tibial is drawn perpendicular to the tibial line measuring the distance between the 2 transverse lines for valgus and varus stress view, respectively.(A) Valgus measurement.(B) Varus measurement.

TABLE 2 Descriptive
Data a a Data are reported as mean 6 SD.